The widespread integration of Large Language Models (LLMs) across various sectors has highlighted the need for empirical research to understand their biases, thought patterns, and societal implications to ensure ethical and effective use. In this study, we propose a novel framework for evaluating LLMs, focusing on uncovering their ideological biases through a quantitative analysis of 436 binary-choice questions, many of which have no definitive answer. By applying our framework to ChatGPT and Gemini, findings revealed that while LLMs generally maintain consistent opinions on many topics, their ideologies differ across models and languages. Notably, ChatGPT exhibits a tendency to change their opinion to match the questioner's opinion. Both models also exhibited problematic biases, unethical or unfair claims, which might have negative societal impacts. These results underscore the importance of addressing both ideological and ethical considerations when evaluating LLMs. The proposed framework offers a flexible, quantitative method for assessing LLM behavior, providing valuable insights for the development of more socially aligned AI systems.

Large Language Models (LLMs) exhibit impressive reasoning abilities, yet their reliance on structured step-by-step processing reveals a critical limitation. While human cognition fluidly adapts between intuitive, heuristic (System 1) and analytical, deliberative (System 2) reasoning depending on the context, LLMs lack this dynamic flexibility. This rigidity can lead to brittle and unreliable performance when faced with tasks that deviate from their trained patterns. To address this, we create a dataset of 2,000 samples with valid System 1 and System 2 answers, explicitly align LLMs with these reasoning styles, and evaluate their performance across reasoning benchmarks. Our results reveal an accuracy-efficiency trade-off: System 2-aligned models excel in arithmetic and symbolic reasoning, while System 1-aligned models perform better in commonsense tasks. A mechanistic analysis of model responses shows that System 1 models employ more definitive answers, whereas System 2 models demonstrate greater uncertainty. Interpolating between these extremes produces a monotonic transition in reasoning accuracy, preserving coherence. This work challenges the assumption that step-by-step reasoning is always optimal and highlights the need for adapting reasoning strategies based on task demands.

Earlier today, Google presented a new vision for its flagship search engine, one that is uniquely tailored to the generative-AI moment. With advanced technology at its disposal, "Google will do the Googling for you," Liz Reid, the company's head of search, declared onstage at the company's annual software conference. Googling something rarely yields an immediate, definitive answer. You enter a query, confront a wall of blue links, open a zillion tabs, and wade through them to find the most relevant information. If that doesn't work, you refine the search and start again.

Despite the remarkable abilities of Large Language Models (LLMs) to answer questions, they often display a considerable level of overconfidence even when the question does not have a definitive answer. To avoid providing hallucinated answers to these unknown questions, existing studies typically investigate approaches to refusing to answer these questions. In this work, we propose a novel and scalable self-alignment method to utilize the LLM itself to enhance its response-ability to different types of unknown questions, being capable of not only refusing to answer but also providing explanation to the unanswerability of unknown questions. Specifically, the Self-Align method first employ a two-stage class-aware self-augmentation approach to generate a large amount of unknown question-response data. Then we conduct disparity-driven self-curation to select qualified data for fine-tuning the LLM itself for aligning the responses to unknown questions as desired. Experimental results on two datasets across four types of unknown questions validate the superiority of the Self-Align method over existing baselines in terms of three types of task formulation.

Despite the great success of large language models (LLMs) in various tasks, they suffer from generating hallucinations. We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes. Specifically, it creates multiple orthogonal bases for modeling truth by incorporating orthogonal constraints into the probes. Moreover, we introduce Random Peek, a systematic technique considering an extended range of positions within the sequence, reducing the gap between discerning and generating truth features in LLMs. By employing this approach, we improved the truthfulness of Llama-2-7B from 40.8\% to 74.5\% on TruthfulQA. Likewise, significant improvements are observed in fine-tuned models. We conducted a thorough analysis of truth features using probes. Our visualization results show that orthogonal probes capture complementary truth-related features, forming well-defined clusters that reveal the inherent structure of the dataset.

The ETHICS dataset consists of five sub-datasets covering different fields of ethics: Justice, Deontology, Virtue Ethics, Utilitarianism, and Commonsense Ethics. The moral judgments were collected via Amazon Mechanical Turk. Please see Hendrycks et al.'s article for more details and examples. GPT-4's performance is much better than that of previous models and suggests that learning to work with common human values is not the hard problem for AI ethics. We found that simple prompt refinements defining the context of the moral judgments and using an embedding to select similar examples from the training set both significantly improved performance. This approach is similar to the "SimPrompting" experiments with GPT-3 [Albrecht et al., 2022].

Though state-of-the-art (SOTA) NLP systems have achieved remarkable performance on a variety of language understanding tasks, they primarily focus on questions that have a correct and a definitive answer. However, in real-world applications, users often ask questions that don't have a definitive answer. Incorrectly answering such questions certainly hampers a system's reliability and trustworthiness. Can SOTA models accurately identify such questions and provide a reasonable response? To investigate the above question, we introduce QnotA, a dataset consisting of five different categories of questions that don't have definitive answers. Furthermore, for each QnotA instance, we also provide a corresponding QA instance i.e. an alternate question that ''can be'' answered. With this data, we formulate three evaluation tasks that test a system's ability to 'identify', 'distinguish', and 'justify' QnotA questions. Through comprehensive experiments, we show that even SOTA models including GPT-3 and Flan T5 do not fare well on these tasks and lack considerably behind the human performance baseline. We conduct a thorough analysis which further leads to several interesting findings. Overall, we believe our work and findings will encourage and facilitate further research in this important area and help develop more robust models.

Can generative AI ChatGPT really serve as a mirror into humanity? Mirror, mirror, on the wall -- humans are the brightest of them all! That isn't of course a proper quotation from the famed Snow White and the Seven Dwarfs, but I opted to leverage the contrivance for a handy purpose. The matter has to do with how humankind sees itself when looking in an all-seeing all-telling mirror. Are we the cat's meow? Do we stand tall above all else? The reason I bring this up has to do with a topic that at first glance might seem afield of the weighty matters underlying how humankind perceives its place in the cosmos. I am going to tie these big-time vexing questions about life, our existence, and humanity all told to the emergence of Artificial Intelligence (AI). Some are insisting that the latest in AI can serve as a mirror into the soul of humanity. Yikes, do we want this? Maybe we won't like what we see. On the other hand, perhaps we have to stiffen our resolve and use AI to see us as we really are. Like a bucket of ice-cold water, AI might be the right thing at the right time to shock us into realizing who we are and where we are going.

The InstructGPT research did recruit 40 contracters to generate a dataset that GPT-3 was then fine-tuned on. But I [Quach] don't think those contractors are employed on an ongoing process to edit responses generated by the model. A spokesperson from the company just confirmed to me: "OpenAI does not hire copywriters to edit generated answers," so I don't think the claims are correct." So the above post was misleading. I'd originally titled it, "Open AI gets GPT-3 to work by hiring an army of humans to fix GPT's bad answers." I changed it to "Interesting questions involving the mix of humans and computer algorithms in Open AI's GPT-3 program." I appreciate all the helpful comments! Stochastic algorithms are hard to understand, especially when they include tuning parameters. I'd still like to know whassup with Google's LaMDA chatbot (see item 2 in this post).

We tackle the task of question generation over knowledge bases. Conventional methods for this task neglect two crucial research issues: 1) the given predicate needs to be expressed; 2) the answer to the generated question needs to be definitive. In this paper, we strive toward the above two issues via incorporating diversified contexts and answer-aware loss. Specifically, we propose a neural encoder-decoder model with multilevel copy mechanisms to generate such questions. Furthermore, the answer aware loss is introduced to make generated questions corresponding to more definitive answers. Experiments demonstrate that our model achieves state-of-the-art performance. Meanwhile, such generated question can express the given predicate and correspond to a definitive answer. 1 Introduction Question Generation over Knowledge Bases (KBQG) aims at generating natural language questions for the corresponding facts on KBs, and it can benefit some real applications. Secondly, the generated questions and answers will be able to augment the training data for QA systems. More importantly, KBQG can improve the ability of machines to actively ask questions on human-machine conversations (Duan et al., 2017; Sun et al., 2018).